Deep Learning Based SWIR Object Detection in Long-Range Surveillance Systems: An Automated Cross-Spectral Approach.

SWIR imaging bears considerable advantages over visible-light (color) and thermal images in certain challenging propagation conditions. Thus, the SWIR imaging channel is frequently used in multi-spectral imaging systems (MSIS) for long-range surveillance in combination with color and thermal imaging to improve the probability of correct operation in various day, night and climate conditions. Integration of deep-learning (DL)-based real-time object detection in MSIS enables an increase in efficient utilization for complex long-range surveillance solutions such as border or critical assets control. Unfortunately, a lack of datasets for DL-based object detection models training for the SWIR channel limits their performance. To overcome this, by using the MSIS setting we propose a new cross-spectral automatic data annotation methodology for SWIR channel training dataset creation, in which the visible-light channel provides a source for detecting object types and bounding boxes which are then transformed to the SWIR channel. A mathematical image transformation that overcomes differences between the SWIR and color channel and their image distortion effects for various magnifications are explained in detail. With the proposed cross-spectral methodology, the goal of the paper is to improve object detection in SWIR images captured in challenging outdoor scenes. Experimental tests for two object types (cars and persons) using a state-of-the-art YOLOX model demonstrate that retraining with the proposed automatic cross-spectrally created SWIR image dataset significantly improves average detection precision. We achieved excellent improvements in detection performance in various variants of the YOLOX model (nano, tiny and x).

Gyroscope-Based Video Stabilization for Electro-Optical Long-Range Surveillance Systems.

Video stabilization is essential for long-range electro-optical systems, especially in situations when the field of view is narrow, since the system shake may produce highly deteriorating effects. It is important that the stabilization works for different camera types, i.e., different parts of the electromagnetic spectrum independently of the weather conditions and any form of image distortion. In this paper, we propose a method for real-time video stabilization that uses only gyroscope measurements, analyze its performance, and implement and validate it on a real-world professional electro-optical system developed at Vlatacom Institute. Camera movements are modeled with 3D rotations obtained by integration of MEMS gyroscope measurements. The 3D orientation estimation quality depends on the gyroscope characteristics; we provide a detailed discussion on the criteria for gyroscope selection in terms of the sensitivity, measurement noise, and drift stability. Furthermore, we propose a method for improving the unwanted motion estimation quality using interpolation in the quaternion domain. We also propose practical solutions for eliminating disturbances originating from gyro bias instability and noise. In order to evaluate the quality of our solution, we compared the performance of our implementation with two feature-based digital stabilization methods. The general advantage of the proposed methods is its drastically lower computational complexity; hence, it can be implemented for a low price independent of the used electro-optical sensor system.